Skip to content

feat(krl): DB-6 Phase A — explain keyword + structured query plan (#34) - #86

Merged
hyperpolymath merged 5 commits into
mainfrom
feat/db6-krl-explain
Jul 28, 2026
Merged

feat(krl): DB-6 Phase A — explain keyword + structured query plan (#34)#86
hyperpolymath merged 5 commits into
mainfrom
feat/db6-krl-explain

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

Summary

Closes the Phase A gap on #34. Before this, EXPLAIN was reachable only over HTTP with raw SQL; the KRL surface the issue actually specifies did not exist — explain appeared in neither the lexer nor the parser, and there was no selectivity model at all, not even the stub the criteria permit.

What's added

  • explain is a reserved word and a statement prefix: explain from … | filter … | take ….
  • It parses to KRLExplainStmt — deliberately a distinct statement type rather than a flag on KRLQueryStmt, so every existing consumer of KRLQueryStmt still means "this query will actually execute" and cannot be handed a plan where it expected rows.
  • ExplainPlan.jl builds the plan shape the acceptance criteria specify — an ordered scan followed by one entry per stage, each filter carrying column, comparator, value, indexed and a selectivity estimate. Conjunctions are split into separate filter operations, which is what makes the DB-7 reorder meaningful later.
explain from knots | filter colouring_count_3 = 9 and crossing < 8 | take 5

{"op" => "scan",   "table" => "knots"}
{"op" => "filter", "column" => "colouring_count_3", "comparator" => "=", "value" => 9,
                   "indexed" => true,  "selectivity_estimate" => 0.5, "selectivity_source" => "stub"}
{"op" => "filter", "column" => "crossing",          "comparator" => "<", "value" => 8,
                   "indexed" => false, "selectivity_estimate" => 0.5, "selectivity_source" => "stub"}
{"op" => "take",   "n" => 5}

Honesty properties (each asserted by a test)

  • Selectivity is the stub the criteria permit pending DB-3 (DB-3: secondary indexes on colouring_count_3 / colouring_count_5 / alexander_polynomial #33) — and says so. Every estimate carries "selectivity_source" => "stub", and plan_is_costed returns false while any stub remains. That is the guard stopping a future DB-7 from silently optimising against invented numbers.
  • indexed is driven by a deliberately short explicit column list. An over-claiming list would make it a fake signal, which is worse than reporting false.
  • A predicate the plan cannot describe is not mis-described. Column-to-column comparisons, nested booleans and calls are reported opaquely ("predicate" => "<expression>", no column key) rather than given a fabricated column.

Verification

25 assertions in server/krl/test/explain_test.jl, wired into the dependency-free CI step alongside the other pure-Julia suites (so it runs before Pkg.instantiate and cannot be masked by a dependency breakage). The existing lexer / parser / sql / seam suites all still pass.

Scope

Phase A only. Phase B (cost-based reordering) is deliberately not attempted — it needs real selectivity, which is DB-3 (#33). The plan returned is execution order as written, with no reordering.

🤖 Generated with Claude Code

hyperpolymath and others added 4 commits July 28, 2026 19:34
…s 12 → 128

main is currently RED: KnotTheory.jl#48 merged, so the conway
@test_broken now reports 'Unexpected Pass' — a hard error, and exactly
the forcing signal the marker existed to produce. This clears it.

- conway(figure_eight) restored to a hard @test. Upstream now
  normalises alexander to Delta(1) = +1 (the Conway normalisation)
  instead of 'leading coefficient positive', which was the wrong
  canonical choice on mixed-sign knots; the value is 1 - z^2 exactly.

- ALEXANDER/CONWAY_MAX_CROSSINGS 12 → 128. The bound of 12 was
  calibrated against the O(n!) cofactor determinant that #48 replaced
  with O(n^3) Bareiss elimination. Re-measured on (2,n)-torus closures
  (the worst case — no arc merging): n=40 → 0.07s, n=100 → 0.10s,
  n=140 → 0.61s, against 6h+ at n=16 before. 128 keeps the worst case
  sub-second while covering every realistic diagram. Jones (20) and
  HOMFLY (15) are unchanged — those are genuinely exponential.

- The guard testset is reworked: with the three bounds now an order of
  magnitude apart, one diagram can no longer exercise them all. It
  tests a 16-crossing diagram (only HOMFLY defers; alexander, conway
  and jones must all be real) AND a 130-crossing diagram (all four
  defer). The second case asserts crossings > ALEXANDER_MAX_CROSSINGS
  explicitly, so raising the bound without raising the fixture fails
  loudly rather than going quiet.

Validated locally against KnotTheory main (a2f2938): the full axioms
suite passes with ZERO broken markers — a first for this suite.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 11/200 count did not move when KnotTheory#48 merged, falsifying the
assumption that the alexander/conway sign convention caused it. Split by
Delta(1) over the same 200-trial corpus:

    links (Delta(1)=0):  11 mismatch, 87 agree
    knots (Delta(1)=±1):  0 mismatch, 102 agree

Knots are now fully crossing-order invariant; the residual is entirely a
multi-component-link defect. Upstream conway_polynomial reconstructs from
an assumed symmetry about exponent 0 that no link satisfies (even span
gives a half-integer centre), so it returns genuinely different
polynomials for the same link — 1 + z^2 vs 1 — rather than a different
unit. Filed as KnotTheory.jl#51 with the worked examples.

Comment updated to cite that evidence instead of the old vague wording.
The marker stays until #51 lands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e instantiate

#83 changed server/Project.toml [sources] from ../../../<name>.jl to
../../<name>.jl but left this workflow symlinking two levels above the
workspace. Every run since has died in 'Instantiate server project' with

    expected package KnotTheory [215268c9] to exist at path
    /home/runner/work/quandledb/KnotTheory.jl

i.e. main has been red for this as well as the conway marker — the path
error hits first, so the axioms suite never even ran.

Symlink now targets $GITHUB_WORKSPACE/../, matching [sources]. Adds a
check that reads server/Project.toml and asserts every [sources] path
resolves to a real directory BEFORE instantiate, so the next drift names
its own cause instead of surfacing as a registry-shaped error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes the Phase A gap on #34. Before this, EXPLAIN was reachable only
over HTTP with raw SQL; the KRL surface the issue actually specifies did
not exist — `explain` was in neither the lexer nor the parser, and there
was no selectivity model at all.

- `explain` is a reserved word and a statement prefix: `explain from … | …`.
- It parses to KRLExplainStmt, deliberately a DISTINCT statement type
  rather than a flag on KRLQueryStmt, so every existing consumer of
  KRLQueryStmt still means "this query will actually execute" and cannot
  be handed a plan where it expected rows.
- ExplainPlan.jl builds the plan shape the acceptance criteria specify:
  an ordered scan + per-stage operations, each filter carrying column,
  comparator, value, `indexed`, and a selectivity estimate. Conjunctions
  are split into separate filter operations, which is what makes the
  DB-7 reorder meaningful later.

Honesty properties, both asserted by tests:

- Selectivity is the stub the criteria permit pending DB-3 (#33), and
  says so: every estimate carries "selectivity_source" => "stub", and
  `plan_is_costed` returns false while any stub remains — so DB-7 cannot
  silently optimise against invented numbers.
- `indexed` is driven by a deliberately short explicit column list. An
  over-claiming list would make it a fake signal, which is worse than
  reporting false.
- A predicate the plan cannot describe (column-to-column, nested
  boolean, call) is reported opaquely rather than given a fabricated
  column.

25 assertions in server/krl/test/explain_test.jl, wired into the
dependency-free CI step alongside the other pure-Julia suites. Existing
lexer/parser/sql/seam suites all still pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread server/krl/ExplainPlan.jl
@gitar-bot

gitar-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

Note

Automatic reviews are paused because your trial's included automatic processing has been used for this period. Upgrade now, or comment "Gitar review" to run a review anytime.
Learn more

Code Review ✅ Approved 1 resolved / 1 findings

Adds the explain keyword and structured query plan generation for KRL queries. However, plan_is_costed can never return true because scan, sort, take, and skip ops default to a "stub" selectivity source.

✅ 1 resolved
Bug: plan_is_costed can never return true

📄 server/krl/ExplainPlan.jl:151-152
plan_is_costed iterates over every op in the plan and uses get(op, "selectivity_source", "stub"), defaulting to "stub" for ops that have no selectivity at all (scan/sort/take/skip). Since those ops never carry a selectivity_source key, the function returns false even once real histograms replace the filter stubs — so it can never fulfill its documented contract ("true only when every selectivity estimate came from real statistics") and DB-7's cost-based path could never activate. Restrict the check to ops that actually carry a selectivity estimate.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@hyperpolymath
hyperpolymath marked this pull request as ready for review July 28, 2026 19:40
@hyperpolymath
hyperpolymath merged commit 9c8d705 into main Jul 28, 2026
3 of 5 checks passed
@hyperpolymath
hyperpolymath deleted the feat/db6-krl-explain branch July 28, 2026 19:40
hyperpolymath added a commit that referenced this pull request Jul 29, 2026
Both were swept into the previous commit by accident. They document the
KRL-side EXPLAIN surface, which lands in #86 — leaving them here would
put documentation of a feature on main ahead of the feature itself.
Restored to main's version; they are committed properly on #86.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
hyperpolymath added a commit that referenced this pull request Jul 29, 2026
…TE refresh (#87)

## Summary

**`main` is red right now** and this fixes it, plus lands two
documentation changes that missed the #85/#86 merges (both PRs were
merged before these commits were pushed).

### 1. The last `@test_broken` retires — main is red until it does

KnotTheory.jl#52 merged the link canonicalisation, so the BR-5
crossing-order marker now reports **"Unexpected Pass"** — a hard error,
and exactly the forcing signal the marker existed to produce.

Measured against KnotTheory `main` after #52: **0 mismatches over 200
trials**, links and knots alike (was 11/200, every one a multi-component
link). Validated at CI's 50 trials — all four BR-5 testsets pass with no
Broken column.

**quandledb now carries no `@test_broken` markers at all.** Every marker
this campaign added was retired by the upstream fix it was tracking,
which is the mechanism working as intended.

### 2. `docs/db-6-explain-strategy.md` — reconciling two surfaces under
one name

The document said "Phase A landed 2026-06-01" while issue #34's Phase A
had never been built. Both statements were true of **different
surfaces**, and conflating them is what made #34 look closable:

| Surface | What | Landed |
|---|---|---|
| SQL-side | `EXPLAIN QUERY PLAN` over read-path SQL, HTTP endpoint |
2026-06-01 |
| KRL-side | `explain` keyword + structured plan with
`indexed`/selectivity | 2026-07-28 (#86) |

Now names both explicitly and documents the KRL surface: syntax, plan
shape, why `explain` parses to its own statement type, why selectivity
is a labelled stub guarded by `plan_is_costed()`, why `INDEXED_COLUMNS`
is deliberately short, and what Phase B actually needs (DB-3, #33).

### 3. `.machine_readable/6a2/STATE.a2ml` — the file agents read said
nothing

Last touched 2026-02-13, with `# No milestones recorded`, `# No blockers
recorded`, `# No actions recorded`. Now carries measured state: phase
`testing`, the DB-6/DB-3 milestone chain, the four live issues (Guix
gate, ruleset `--admin`, doc drift, ungated Idris2/Zig/AffineScript
tiers) each with an evidence path, ordered next actions with owners, and
`zero-known-broken-markers = true`.

`last-result` is deliberately **`warn`, not `pass`**: every gate the
repo owns is green, but the estate-wide Guix policy check is red and is
not green-able here in good faith (see
`dev-notes/guix-policy-gate-advice-2026-07-28.md` — the gate is
presence-only, so a stub `guix.scm` would buy green with a lie).
Reported rather than suppressed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant